Modeling and simulation of adaptive multimodal optical sensors for target tracking in the visible to near infrared

This work investigates an integrated aerial remote sensor design approach to address moving target detection and tracking problems within highly cluttered, dynamic ground-based scenes. Sophisticated simulation methodologies and scene phenomenology validations have resulted in advancements in artificial multimodal truth video synthesis. Complex modeling of novel micro-opto-electro-mechanical systems (MOEMS) devices, optical systems, and detector arrays has resulted in a proof of concept for a state-of-the-art imaging spectropolarimeter sensor model that does not suffer from typical multimodal image registration problems. Test methodology developed for this work provides the ability to quantify performance of a target tracking application with varying ground scenery, flight characteristics, or sensor specifications. The culmination of this research is an end-to-end simulated demonstration of multimodal aerial remote sensing and target tracking. Deeply hidden target recognition is shown to be enhanced through the fusing of panchromatic, hyperspectral, and polarimetric image modalities. The Digital Imaging and Remote Sensing Image Generation model was leveraged to synthesize truth spectropolarimetric sensor-reaching radiance image cubes comprised of coregistered Stokes vector bands in the visible to near-infrared. An intricate synthetic urban scene containing numerous moving vehicular targets was imaged from a virtual sensor aboard an aerial platform encircling a stare point. An adaptive sensor model was designed with a superpixel array of MOEMS devices fabricated atop a division of focal plane detector. Degree of linear polarization (DoLP) imagery is acquired by combining three adjacent micropolarizer outputs within each 2x2 superpixel whose respective transmissions vary with wavelength, relative angle of polarization, and wire-grid spacing. A novel micromirror within each superpixel adaptively relays light between a panchromatic imaging channel and a hyperspectral spectrometer channel. All optical and detector sensor effects were radiometrically modeled using MATLAB and optical lens design software. Orthorectification of all sensor outputs yields multimodal pseudonadir observation video at a fixed ground sampled distance across an area of responsibility. A proprietary MATLAB-based target tracker accomplishes change detection between sequential panchromatic or DoLP observation frames, and queries the sensor for hyperspectral pixels to aid in track initialization and maintenance. Image quality, spectral quality, and tracking performance metrics are reported for varying scenario parameters including target occlusions within the scene, declination angle and jitter of the aerial platform, micropolarizer diattenuation, and spectral/spatial resolution of the adaptive sensor outputs. DoLP observations were found to track moving vehicles better than panchromatic observations at high oblique angles when facing the sensor generally toward the sun. Vehicular occlusions due to tree canopies and parallax effects of tall buildings significantly reduced tracking performance as expected. Smaller MOEMS pixel sizes drastically improved track performance, but also generated a significant number of false tracks. Atmospheric haze from urban aerosols eliminated the tracking utility of DoLP observations, while aerial platform jitter without image stabilization eliminated tracking utility in both modalities. Wire-grid micropolarizers with very low VNIR diattenuation were found to still extinguish enough cross-polarized light to successfully distinguish and track moving vehicles from their urban background. Thus, state-of-the-art lithographic techniques to create finer wire-grid spacings that exhibit high VNIR diattenuation may not be required

Dynamic Image Simulations for Adaptive Sensor Performance Predictions

ABSTRACT Comprehensive simulation of multispectral/hyperspectral synthetic imagery captured from a moving aerial platform is necessary for predicting real-life performance of adaptive sensor models. Physics-based modeling and simulation tools are required to synthesize sensor-reaching radiance imagery of an urban scene while incorporating moving vehicular targets, moving platform position & orientation with uncertainty, atmospheric conditions, and time-of-day effects. Trade studies of design parameters can be conducted to compare sensor output image and spectral quality. Orthorectification of direct detector imagery using a digital elevation map can produce registered imagery for targettracking applications. This paper describes a novel combination of these modeling tools to support end-to-end adaptive sensor design studies

Simulation of practical single-pixel wire-grid polarizers for superpixel stokes vector imaging arrays Optical Engineering

Abstract. An optical tracking sensor that produces images containing the state of polarization of each pixel can be implemented using individual wire-grid micropolarizers on each detector element of a solid-state focal plane array. These sensors can significantly improve identification and tracking of various man-made targets in cluttered, dynamic scenes such as urban and suburban environments. We present electromagnetic simulation results for wire-grid polarizers that can be fabricated on standard imaging arrays at three different technology nodes (an 80-, 250-, and 500-nm pitch) for use in polarization-sensitive detector arrays. The degradation in polarizer performance with the larger pitch grids is quantified. We also present results suggesting the performance degradation is not significant enough to affect performance in a man-made vehicle-tracking application